Staging links:
https://redis.io/docs/staging/DOC-5003/commands/json.debug-memory/
https://redis.io/docs/staging/DOC-5003/develop/data-types/json/ram

@eranhd can you help here?

@dwdougherty no, it could not be 8 + 8.

The size of an empty array is 8 because it consists only of a pointer to a static "null" array. if it is non-empty, we need to store the additional metadata info (size, capacity) which is 16 more bytes.

Why it's 64 and not 32, I'll investigate. (Although it might just be as simple as the array having extra unused capacity.)

Yes, the array is created with default capacity 4. I did not investigate why. That leaves us with 56 bytes of used memory.

The additional 8 bytes comes from, as far as I can tell, a mistaken double counting of the string. It is being counted once when accounting for the used capacity of the array, and being counted again when recursively counting the memory consumed by the values stored in the array.

@LiorKogan @eranhd this should probably be fixed.

The expected behavior should be

127.0.0.1:6379> json.set a . '["", ""]'
OK
127.0.0.1:6379> json.debug memory a
(integer) 56
127.0.0.1:6379> json.set a . '["", "", ""]'
OK
127.0.0.1:6379> json.debug memory a
(integer) 56
127.0.0.1:6379> json.set a . '["", "", "", ""]'
OK
127.0.0.1:6379> json.debug memory a
(integer) 56
127.0.0.1:6379> json.set a . '["", "", "", "", ""]'
OK
127.0.0.1:6379> json.debug memory a
(integer) 88

Hi @ephraimfeldblum. Would you please break down the first example, '[""]' for me byte for byte? I want to make sure I have it right in the JSON memory page.

BTW, in my latest commit, I re-ran all the examples using the 8.0-rc1 bits.

Absolutely,

All JSON values are 8 bytes because the type used to represent them is a thin wrapper around a pointer. The information about what kind each value is is stored in the lower bits of the pointer. They are guaranteed to be zero because of alignment restrictions, therefore we can use them to store some entropy.

For some kinds of JSON values, that's all there is to it. Nulls and bools require not even that. Small integer values are already stored in static memory because we expect them to be frequently used, so they also take no more memory than the initial 8 bytes. Similarly, empty strings, arrays, and objects do not require any bookkeeping, and they can point to a static "null" string, array, or object as appropriate. You can create a database with as many of any of those as you like, and each instance will consume only the initial 8 bytes with no added allocations beyond that.

Once an array value (in this example) is filled with subvalues, extra metadata is required to track the allocations being done. 8 bytes are used each to quantify the allocated capacity and the real size of the array, totaling 16 bytes. So an array with only one contained value consumes: 8 bytes of the initial array value pointer + 16 bytes of metadata + and 8 × capacity bytes of allocated memory. By default, the initial capacity is 4. In total, that comes out to 56 bytes. Since we don't reallocate memory until the initial capacity is filled, we can fill such an array with up to 4 values and the memory consumed does not change. Once the current capacity is insufficient to fit a new value, the array reallocates to double* its capacity. An array with 5 elements will have a capacity of 8, therefore consuming 8 + 16 + 8×8 = 88 bytes.

*Since this operation is potentially expensive, it is done geometrically, not linearly. That way its cost may be amortized over many insertions.

@ephraimfeldblum thank you!

@dwdougherty I believe we can add a paragraph along these lines, followed by @ephraimfeldblum 's text:

Redis' internal JSON encoding is optimized for both performance and memory footprint. Due to such optimizations, the reply of JSON.DEBUG MEMORY could be nonintuitive. For example...

Thank you, @ephraimfeldblum! Much appreciated!

Thanks, @andy-stark-redis !